Digital readout method and apparatus

ABSTRACT

Autonomously operating analog to digital converters are formed into a two dimensional array. The array may incorporate digital signal processing functionality. Such an array is particularly well-suited for operation as a readout integrated circuit and in combination with a sensor array, forms a digital focal plane array.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/651,040, filed Jul. 17, 2017, entitled “DIGITAL READOUT METHOD ANDAPPARATUS,” which is a continuation of U.S. application Ser. No.15/178,439, now U.S. Pat. No. 9,712,771, filed Jun. 9, 2016, entitled“DIGITAL READOUT METHOD AND APPARATUS,” which is a continuation of U.S.application Ser. No. 14/563,271, now U.S. Pat. No. 9,385,738, filed Dec.8, 2014, entitled “DIGITAL READOUT METHOD AND APPARATUS,” which is acontinuation of U.S. application Ser. No. 13/445,433, now U.S. Pat. No.8,933,832, filed Apr. 12, 2012, entitled “DIGITAL READOUT METHOD ANDAPPARATUS,” which is a continuation of U.S. application Ser. No.11/415,007, now U.S. Pat. No. 8,179,296, filed May 1, 2006, entitled“DIGITAL READOUT METHOD AND APPARATUS,” which claims the prioritybenefit of U.S. Application No. 60/722,741, filed Sep. 30, 2005,entitled “ALL-DIGITAL READOUT INTEGRATED CIRCUIT ARCHITECTURE FOR VERYLARGE, HIGH FRAME RATE, AND WIDE DYNAMIC RANGE INFRARED IMAGING ARRAYS.”Each of these applications is hereby incorporated herein by reference intheir entireties.

GOVERNMENT INTERESTS

This work was made with government support under contract no.F19628-00-C-0002 awarded by the U.S. Air Force. The Government hascertain rights in this invention.

FIELD OF THE INVENTION

The invention relates to analog to digital converters (ADCs) and, inparticular, to collections of ADCs.

BACKGROUND

ADCs are employed in a variety of applications. In particular, highperformance focal plan array (FPA) applications require wide-areacoverage, high signal-to-noise-ratios (SNR), high spatial resolution,and high frame rates in various combinations. Conventional FPAs are notparticularly well-suited to satisfying combinations of the aboverequirements. Conventional FPAs typically provide analog readouts, withthe analog signals generated at the pixel level converted to digitalsignals “off chip.” Once converted off-chip, the digital signals may beprocessed according to the demands of a particular application. Specificanalog designs can target (and possibly achieve) one or morerequirement, but fail when simultaneously targeting the most aggressivedesign parameters for imaging applications, such as long-wave infraredimaging (LWIR) applications.

Fundamental limitations on achievable well depth (with concomitantlimitations on capacitor size), and the readout noise floor, limitpractical scalability of conventional designs. Capacitor sizelimitations require unnecessarily high frame rates to avoid saturatingpixels. Electronics noise and ringing limit the amount of data that canbe transmitted on a single output lap to maintain the needed SNR anddynamic range. Attempting to scale conventional analog technology tomeet the most demanding requirements leads to a high-power-consumptionFPA with many data output taps. This in turn leads to a large, massive,and complex sensor system.

ADCs that are capable of converting a number of signals and acommunication structure for manipulating the resulting digital counts inreal-time to achieve signal processing functionality, would therefore behighly desirable and may find application in a number of systems,including focal plane arrays.

An apparatus in accordance with the principles of the present inventionincludes a two-dimensional array of analog to digital converters (ADCs).In an illustrative embodiment, the ADCs are all self-contained. That is,in such an embodiment, each of the ADCs is capable of operating withoutsignaling, such as an analog ramp signal, or analog reference signalsupplied by an outside source. Each of the ADCs within the array mayinclude circuitry to convert a current mode signal to a voltage signalfor conversion. In such an embodiment, a capacitor may be employed tointegrate charge from the current mode signal and the capacitor and ADCarchitecture may be selected to determine the least significant bit ofeach of the ADCs. In such an embodiment, a counter may be employed todetermine the most significant bit of each of the ADCs.

An ADC array in accordance with the principles of the present inventionmay include circuitry that permits the orthogonal transfer of conversionresults throughout the array. Such an array may include circuitry thatpermits the array to convert the plurality of analog signals to digitalsignals, and then perform digital processing on the resulting digitalsignals. The processed signals may then be sent, for example, to ananalyzer for computation and analysis. In accordance with the principlesof the present invention, digital signal processing functions includedwithin the ADC array may include digital filtering, such as spatial ortemporal filtering, autonomous digital threshold detection, time-domainfiltering, including high-pass or low-pass filtering, and datacompression, using, for example, Decimation.

Additionally, an ADC array in accordance with the principles of thepresent invention may be configured to accept and convert analog signalsthat are spatially mapped to the arrangement of ADCs within the array.The spatial mapping may be, for example, a one-to-one mapping, withsignals arriving at the top left ADC within the array originating at acorresponding location within an array of signals, the signal arrivingat the bottom right ADC within the array originating at a correspondinglocation within an array of signals, and so on. In an integrated circuitembodiment, an entire ADC array may be implemented using a silicon CMOSprocess, for example.

An ADC array in accordance with the principles of the present inventionmay be employed as a readout integrated circuit that operates inconjunction with, for example, a photosensor array. In such anembodiment, each of the ADCs within the array may occupy no more areathan the area consumed by each of the corresponding photosensors. Areadout integrated circuit that employs an ADC array in accordance withthe principles of the present invention may foe combined with aphotosensor array, using hybrid techniques, such as bump-bonding, forexample, to form a novel focal plane array.

In an Illustrative embodiment an all-digital readout Integrated circuit(ROIC) in accordance with the principles of the present invention may beused in conjunction with a cryogenically cooled infrared detector array,with connections between the detector array and the ROIC made via indiumbump bonding. The hybrid device thus formed is referred to herein as adigital focal plane array (DFPA). In an illustrative embodiment, thedetector array senses incoming optical radiation in the infrared regionof the spectrum (2-20 microns) using photodiodes to create currents thatare proportional to the optical radiation impinging on the photodiodes.That is, each photodiode (also referred to herein as a pixel) in thedetector array produces a current that is proportional to the photonflux impinging upon it. Each photodiode in the array has associated withit a unit cell in the ROIC. The current in each photodiode is collectedin the photodiode's associated unit cell within the ROIC. The unit cellelectronics integrate the charge and produces, via an analog to digitalconverter (ADC), a digital number (DN) that is proportional to the totalcharge accumulated over the frame period. In this illustrativeembodiment, the DN for each pixel is then shifted to the edge of theROIC and multiplexed with other DNs associated with other pixels fortransfer off the array. By digitizing the signal while photoelectronsare being collected, rather than after charge accumulation, the need forlarge charge storage capacitors and highly linear analog electronics canbe eliminated. The power dissipation and noise problems associated witha conventional, analog readout, approach are also greatly reduced.Additionally, this approach permits operation with circuitry thatoperates from a lower level power supply, because the dynamic rangerequirements associated with conventional systems needn't be maintained.Permitting operation with tower-level power supplies permits the use ofIntegrated Circuit processes that offer much smaller feature sizes,thereby further enabling the ADC and readout circuitry to be packedwithin an area less than or equal to the area consumed by the associateddetector pitch, also referred to as the pixel pitch. Simplifying theunit cell preamplifier offers considerable power savings for largearrays.

In this illustrative embodiment, the capacitor size defines the leastsignificant bit of the ADC. In this way, the size of the capacitor maybe kept to a minimum, thereby significantly reducing the area requiredfor the analog to digital conversion. In this illustrative embodiment,the analog to digital conversion is achieved via a voltage-to-frequencyconverter in which a predetermined amount of photocurrent charges thecapacitor to a level that produces an output pulse and resets thecapacitor. The output pulses are counted and the count in a given timeperiod corresponds to the amount of photocurrent and, correspondingly,the light flux impinging on the associated photodiode. In this way, theillustrative embodiment of a DFPA in accordance with the principles ofthe present invention digitizes the signal while photoelectrons arebeing collected, rather than after charge accumulation.

A system and method in accordance with the principles of the presentinvention may be employed to form a DFPA that includes a Nyquist-rateADC formed wholly within the area of the ADC's associated detector, orpixel pitch. In such an embodiment, each of the ADCs may operateindependently of the other ADCs associated with other photodiodes.

An ADC array in accordance with the principles of the present inventionmay be particularly well-suited for use in an imaging system. In such animplementation, an imaging system might employ an ADC array inaccordance with the principles of the present invention in conjunctionwith a sensor array, such as a photosensor array, to gather and processanalog signals. The analog signals subject to processing in such animplementation may represent electromagnetic radiation characterized byany wavelength, and need not be limited to signals that mightconventionally be associated with “imaging.”That is, in addition tosignals that represent photon flux, the signals may, directly, orindirectly, represent, chemical or biological content, for example. Suchan imaging system may employ an ADC array in accordance with theprinciples of the present invention to, in addition to converting analogsignals to digital, perform digital signal processing operations on theconverted signals. Further computations and analysis may be performed bythe imaging system on the output of the ADC array. An imaging system inaccordance with the principles of the present invention would beparticularly suited to use in industrial inspection, surveillance,process control, biological research, chemical research,pharmaceuticals, medical imaging, remote sensing, and astronomy, forexample.

A system in accordance with the principles of the present inventionintegrates a focal plane array with digital readout circuitry. In anillustrative embodiment, the focal plane array includes an army ofoptical diodes, with each diode defining a pixel. In accordance with theprinciples of the present invention, circuitry is included in the focalplane array to convert the signal from each diode to a digital signal.Each pixel includes an analog to digital converter (ADC) that is formedwithin the same area as the photodiode. In an illustrative embodiment,the digital signals produced by the ADCs at each photodiode are routedoff- chip. The density of the photodiode array is not limited by the ADCand associated circuitry at each pixel location. Consequently, extremelyhigh-resolution imaging may be achieved by a digital focal plane array(DFPA) In accordance with the principles of the present invention.

An apparatus and method in accordance with the principles of the presentinvention may also be employed by a DFPA to incorporate on-chipprocessing to further enhance the DFPA's performance. Such processingmay include: digital spatial/temporal filtering that may be implementedby pixel binning whereby image resolution may be traded for dynamicrange and/or frame rate; autonomous digital threshold detection that maybe used in launch detection or flash detection, for example; time domaindigital filtering, such as high-pass or low-pass filtering that may beemployed to identify fast- or slow-moving objects; time domain imagedetection and jitter compensation that, for example, that shifts digitalrow data synchronously with ground speed and provides imagestabilization for field of view jitter; data compression to reduce rawdata rate using, for example, decimation; and the use of edgeenhancement, such as octonet edge enhancement to aid in featureidentification.

A sensor that employs a DFPA in accordance with the principles of thepresent invention may operate in a variety of modes. In an unmannedaerial vehicle (UAV) application, for example, such a sensor may operatein: a high-resolution day/night panchromatic surveillance mode; a lowresolution day/night rapid panchromatic battlefield monitoring,flash/launch detection mode; a low resolution day/night hyper-spectraleffluence classification and atmospheric condition mode; and a timedomain imaging (TDI) mode for “push broom” terrain mapping. An apparatusand method in accordance with the principles of the present inventionmay be particularly well-suited for use in a large format digital focalplane array (DFPA). Such a DFPA could provide high-resolution wide-areacoverage and would thereby find application in air and spaceintelligence, surveillance, and reconnaissance (ISR). In such anapplication the DFPA could drastically reduce the number of assetsrequired for global persistent day/night surveillance and launchdefection and fully utilize diffraction-limited resolution. The DFPAcould also be employed in atmospheric monitoring, providing the abilityto detect the initiation of severe weather events on a global scale,thereby lengthening warning times that might allow those in the paths ofstorms to avoid or ameliorate disaster. A DFPA in accordance with theprinciples of the present invention may also be used in the detectionand identification of chemical agents, for example, in the search forweapons, including weapons of mass destruction. Such detection andidentification of chemical agents may also be employed in chemical,medical, or industrial systems in large-scale industrial inspection andmassive parallel molecular process monitoring applications.

An ADC array in accordance with the principles of the present inventionmay incorporate digital signal processing circuitry in the array. In anillustrative embodiment, an array includes an up/down counter at eachADC location and circuitry for the orthogonal shifting of the digitaloutput of each ADC. By controlling the accumulation time, the number anddirection of counts, and the number and direction of shifts, the ADCarray itself may be employed as an array-wide digital signal processor.

An imaging system that incorporates an ADC array in accordance with theprinciples of the present invention may be configured to operate, forexample, as a camera (in any of a number of electromagnetic bands,including, but not limited to, the visual and infrared) as asurveillance system, as a robotic control system, or as a chemical orbiological detection or identification system.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and further features, aspects, and advantages of the inventionwill be apparent to those skilled in the art from the following detaileddescription, taken together with the accompanying drawings in which:

FIG. 1 is a block diagram of a two-dimensional analog to digitalconverter (ADC) array in accordance with the principles of the presentinvention;

FIG. 2 is a block diagram of an illustrative embodiment of an ADC suchas may be employed in an ADC array in accordance with the principles ofthe present invention;

FIG. 3 is a schematic diagram of an illustrative embodiment of adetector in combination with a voltage to frequency converter, such asmay be employed by an ADC array in accordance with the principles of thepresent invention;

FIG. 4 is a block diagram of an illustrative ADC cell such as may beemployed by an ADC array in accordance with the principles of thepresent invention;

FIG. 5 is a schematic diagram that illustrates the interconnection pathsof ADC within an ADC array in accordance with the principles of thepresent invention;

FIG. 6 is a block diagram of an ADC such as maybe incorporated in an ADCarray in accordance with the principles of the present invention;

FIGS. 7A and 7B are block diagrams of data handling blocks such as maybe employed in an ADC array in accordance with the principles of thepresent invention;

FIG. 8 is a block diagram of and imaging system in accordance with theprinciples of the present invention; and

FIG. 9 is a flow chart of an illustrative digital signal processingoperation such as may be executed by an ADC array in accordance with theprinciples of the present invention.

DETAILED DESCRIPTION

The block diagram of FIG. 1 illustrates an apparatus in accordance withthe principles of the present invention that includes a two-dimensionalarray 100 of analog to digital converters (ADCs), ADC.sub.ij (where i=1through n and j-1 through m). In this illustrative embodiment, the ADCsare all self-contained. That is, in such an embodiment, each of the ADCsis capable of operating without signaling, such as an analog rampsignal, or analog reference signal supplied by an outside source. Eachof the ADCs within the array may include circuitry to convert a currentmode signal to a voltage signal for conversion. In such an embodiment, acapacitor may be employed to integrate charge from the current modesignal and the capacitor and ADC architecture may be selected todetermine the least significant bit of each of the ADCs. In such anembodiment, a counter may be employed to determine the most significantbit of each of the ADCs.

The block diagram of FIG. 2 provides a more detailed view of anindividual ADC, such as may be found employed within an ADC array inaccordance with the principles of the present invention. An amplifier200 (which is optional, depending upon the quality of the signal to beconverted) accepts an analog signal, AN, for conversion, amplifies andconditions the signal, AN, and provides the conditioned signal to avoltage to frequency converter 202. In illustrative embodiments theamplifier 200 may be a direct injection, buffered direct injection,source-follower, or transimpedence amplifier, for example. Thevoltage-to-frequency converter 202 converters the voltage signal fromthe amplifier to a serial digital signal, the frequency of which isrepresentative of the voltage input. The digital output of thevoltage-to-frequency converter is routed to a counter 204, where thedigital stream is counted. The resulting count, DN, is a digital signalthe magnitude of which is representative of the magnitude of the inputanalog signal AN. In an illustrative embodiment, the each ADC in thearray 100 includes a shift register 206 that may be employed to shiftthe digital output, DN, to other ADCs within the array 100 and/oroutside of the array, for further processing and analysis, for example.

The schematic diagram of FIG. 3 provides a more detailed illustration ofan amplifier 200 and voltage-to-frequency converter 202, such as may beemployed in an ADC array 100 in accordance with the principles of thepresent invention. Such an embodiment may be used in conjunction with aphoto-sensor array to form a readout integrated circuit, or, if thephoto-sensor array is also incorporated, a digital focal plane array, ina photo-sensor array embodiment, a photo-sensor would produce a currentsignal AN, that is representative of the photon flux impinging upon thephoto-sensor. In this illustrative embodiment, a photo-diode PD producesa current in response to electromagnetic radiation impinging upon thephoto-diode PN. As is known in the art, various types of photo-sensorsmay be employed to sense energy of different electromagneticwavelengths. Current from the photo-diode PD is amplified by the passtransistor PT. The amplified current from the pass transistor is fed tothe capacitor, CAP, where the charge is accumulated, thereby increasingthe voltage on the capacitor CAP.

The capacitor voltage signal is routed to the voltage to frequencyconverter 202, at the input to the first of four inverters, INV1, INV2,INV3, and INV4, connected in series. When the voltage on the capacitorCAP reaches the threshold voltage of the inverter INV1, the output stateof INV1 switches (from “LOW” to “HIGH” in this illustrative embodiment).The subsequent inverters in the string, INV2 through INV4, also switchand the output, PULSE, switches state (from “LOW” to “HIGH” in thisillustrative embodiment). When the signal PULSE goes “HIGH,” it turns onthe drain transistor, DT, which drains the capacitor, CAP. When thevoltage on the capacitor, CAP, is drained below the threshold voltage ofthe inverter INV1, the inverter INV1, as well as subsequent inverters inthe chain (e.g., INV2, INV3, INV4), change state, once again. The resultof charging and discharging the capacitor, CAP, is, therefore, in thisillustrative embodiment, a positive-going output pulse. As photonscontinue to impinge upon the photodiode PD, the capacitor will continueto charge to a voltage above the threshold voltage of the inverter INV1,switch the state of the inverters, be discharged by drain transistor DT,and, consequently, produce more output pulses. The rate at which photonsimpinge upon the photodiode is proportional to the current produced bythe photodiode and the rate at which the capacitor, CAP, is charged isalso, therefore related to the rate at which photons impinge upon thephotodiode PD. The rate at which pulses are produced is proportional tothe rate at which the capacitor is charge and, therefore, the pulse rateoutput is proportional to the rate at which photons impinge upon thephotodiode. The inverters INV2 through INV4 also provide pulse-shapingfor the output signal, PULSE. In an illustrative embodiment,photocurrent is integrated onto the capacitor, CAP, until the thresholdof the first stage inverter INV1 is reached. In this embodiment, theintegration capacitor, CAP, is in the single-digit femtofarad range tomeet a 10 kHz frame rate requirement with the appropriate inputphotocurrent. The capacitance value may be achieved, for example, byusing the parasitic capacitance of the first inverter gate. In someapplications, in the visible range, for example, it may be advantageousto charge the capacitor CAP at a higher rate for a given photo flux. Anavalanche photodiode may be employed in order to charge the capacitor ata greater rate for a given photon flux. Additionally, the “effectivecapacitance” of the capacitor CAP may be reduced, allowing a smallerphoton flux to switch the first inverter stage, by discharging acapacitor to a predetermined threshold level. A current mirror with gaincan be used as well.

The block diagram of FIG. 4 provides a more detailed view of an ADC 400and peripheral circuitry, such as may be employed in an array 100 inaccordance with the principles of the present invention. In thisillustrative embodiment, an HgCdTe photo-detector, such as may beemployed for sensing electromagnetic radiation in the infrared region ofthe spectrum, is bonded via bump-bond 406 to the input of the ADC 400.In this embodiment, an output multiplexor 404 is employed to multiplexthe results of sixteen rows of ADCs. The Input amplifier 408 andvoltage-to-frequency converter 410 may be as described in the discussionrelated to FIG. 3, for example. In this illustrative embodiment, thepulse stream from the voltage to frequency converter 410 is fed to aripple counter 412, which counts the pulses representative of the inputphoton flux impinging upon the photodetector 402. The results of theripple counter may be shifted in parallel into a pixel output register414, which, in turn may be employed to shift the results out of thearray, through the output multiplexor 404, for example. The pixel outputshift register 414 may also receive the output of other ADCs through thePIXIN input to the shift register 414. In this way, the count resultsfrom a row of ADCs may be shifted out through the output multiplexor404.

An ADC array 100 may be employed, for example, in high performancelong-wave infrared (LWIR) imaging applications that demand wide areacoverage, high SNR, and high spatial resolution. In such an embodiment,a sensor array equipped with a large format, cryogenically cooledHgl-xCdxTe focal plane array (FPA) with small pixels would supply analogcurrent-mode signals to corresponding ADCs within the ADC array 100. Theminimum useful pixel size in such a sensor array will ultimately bedriven by the optical system. Advanced spectral sensors also demand veryhigh frame rates to collect hundreds of channels in a short period oftime. As previously described, conventional (analog) FPAs are not wellsuited to meet all of these requirements. Specific analog designs cantarget (and possibly achieve) one or more requirement, but fail whensimultaneously targeting the most aggressive design parameters for LWIRapplications. Fundamental limitations on achievable well depth(capacitor size) and the readout noise floor limit practical scalabilityof conventional designs. Capacitor size limitations requireunnecessarily high frame rates to avoid saturating pixels. Electronicsnoise and ringing limit the amount of data that can be transmitted on asingle output tap to maintain the needed SNR and dynamic range.Attempting to scale conventional analog technology to meet the mostdemanding requirements leads to a high-power FPA with many data outputtaps. This in turn leads to a large, massive, and complex sensor system.A digital focal plane array that employs an ADC array in accordance withthe principles of the present invention may exploit commerciallyavailable, low voltage, and deeply scaled sub-micron CMOS processes,and, thereby, significantly reduce costs, in addition to providingsuperior performance. In an illustrative embodiment, such as describedin the discussion related to FIG. 3, the charge accumulation capacitor,CAP, effectively defines the magnitude of the least significant bit ofthe ADC and the analog signal is digitized “on the fly,” asphotoelectrons generate charge, rather than waiting for chargeaccumulate on a large capacitor which would define the full-scale valueof the ADC. By digitizing the signal while photoelectrons are beingcollected, rather than after charge accumulation, the need for largecharge storage capacitors and highly linear analog electronics iseliminated. The power dissipation and noise problems associated with theanalog readout approach are also greatly reduced.

That is, for example, in a conventional, LWIR analog, focal plane arraya Hgl-xCdxTe photodiode array may be mated to a Silicon (Si) readoutintegrated circuit (ROIC). Photons absorbed within the active region ofthe photodiode detector are converted to electrons, producing aphotocurrent. A conventional FPA Integrates the current during a frameperiod onto a large capacitor, producing an analog voltage. The voltageproduced on each capacitor, within each pixel, is proportional to thelight intensify incident on the pixel. At the end of a frame period, oneof several possible methods is used to transfer the voltage value foreach pixel to an analog multiplexer and output driver and the capacitoris reset. Off-chip electronics condition the resulting analog datastream for quantization by an A/D converter, in this architecture, thecapacitor size determines the most significant bit (MSB), and theoff-chip electronics determine the least significant bit (LSB) of thesensor.

In a digital focal plane array that employs an ADC array in accordancewith the principles of the present invention, the photocurrent drives avoltage-to-frequency (V/F) converter. The input of the V/F converterconsists of a very small capacitor, which integrates the photocurrent.When the voltage reaches a pre-defined threshold level, a counter withinthe pixel is incremented and the capacitor is reset. The counter isincrementally increased throughout the entire frame period. Noadditional off-chip electronics are needed. At the end of a frameperiod, the digital counts for each pixel are transferred to a digitalmultiplexer and output driver for readout. In this architecture, thecounter size determines the MSB and the capacitor size determines theLSB of the sensor. As previously described, a constituent ADC includes apre-amplifier 408, voltage-to-frequency converter 410, sequential ornon- sequential counter 412, and shift register 414. The shift register414 may be employed for “snapshot” imaging in high backgroundapplications. Shift registers 414 may be serially connected to adjacentADCs in each row to read out data bits, in snapshot mode, counters 412within each ADC. In the array 100 can operate while data from theprevious frame is read from the array 100. In low background, longintegration applications, the ripple counter 412 can be configured tocount or shift values to adjacent pixels. In this configuration, thereadout operates in a burst mode with little loss of signal. Significantreduction in ADC area can be achieved when the readout can operate inburst mode. In this illustrative embodiment, a ripple counterconfiguration was chosen over a synchronous counter because of its lowerpower consumption; every bit of a synchronous counter would be clockedat every V/F converter pulse. A ripple counter only clocks an average oftwo bits per pulse. The dynamic D flip-flop structures were built usingtrue-single phase clock (TSPC) logic. Other dynamic D flip-flop designsmay be employed, although they will, typically, consume more area. Usingthis design structure, one may implement the design using 12 transistorsper register cell, making layout of an area-constrained design feasible.The three logic control lines handle the reset of the ripple counter,the load of the shift registers, and the clocking of the shift registersto output the data from the previous integration period.

As previously described, photo-current from a detector, such as detector402, drives the voltage-to-frequency converter through a preamplifier408. A wide variety of pre-amplification techniques are compatible withan ADC array in accordance with the principles of the present invention(and readout integrated circuit and digital focal plane array thatemploy such an ADC array). Since the preamp is reset on each LSB,linearity is not a major issue as long as it is stable. In thisillustrative embodiment, the voltage-to-frequency converter produces apulse stream that is used as an asynchronous clock to drive the counter.At the end of a frame period, the digital number in the counter 412 istransferred to the shift register 414 and then to a 16:1 digitalmultiplexer 404 located at the edge of the unit cell array. In thisillustrative embodiment, the multiplexer maximum output data rate (2.5Gbps) was chosen for compatibility with off-the-shelf receivingelectronics.

The signal to noise ratio achievable with a digital focal plane array inaccordance with the principles of the present invention can becalculated from Eq. 1. The effective number of bits (ENOB) is aconvenient figure of merit for comparing the digital focal plane arrayperformance to existing sensor systems and commercial ADC products. TheENOB describes the SNR of the system, under stated sampling conditions,relative to the quantization noise of an ideal A/D converter. The ENOBspecification for a real A/D converter is always lower than the maximumhit depth.

$\begin{matrix}{{SNR} = \frac{1}{\sqrt{\frac{1}{12\; N^{2}} + {\frac{1}{{NC}^{2}V^{2}}\left\lbrack {{CVq} + {kTC} + {\frac{e_{n}^{2}}{R_{d}^{2}}\frac{t}{2N}}} \right\rbrack}}}} & {{Eq}.\mspace{14mu} 1} \\{{E\; N\; O\; B} = {{\log_{2}({SNR})} - 1.79}} & {{Eq}.\mspace{14mu} 2}\end{matrix}$

Where N is the decimal count value read-out from the pixel, C is theeffective input capacitance into the V/F converter, V is the thresholdvoltage of the V/F converter, q is the electronic charge unit, k isBoltzmann's constant, T is the temperature, e.sub.n is the inputreferred voltage noise density of the preamp, R.sub.d is the detectorshunt resistance, and t is the frame integration time. The modelconsiders quantization, kTC (associated with resting a capacitor),preamp, and shot noise.

As illustrated in the block diagram of FIG. 5, an ADC array 500 inaccordance with the principles of the present invention may includecircuitry that permits the orthogonal transfer of conversion resultsthroughout the array. Each block within the illustrative array, such asblock 502, represents an ADC. Digital results from each ADC may betransferred through a column transfer 504 and/or through a row transfer506. In this manner, any result from any ADC within the array 500 may betransferred to any other ADC within the array 500; such transfercapability is what is referred to herein as orthogonal transfer. Such anorthogonal transfer capability provides an element of digital signalprocessing operation to the array 500 that permits the array to converta plurality of analog signals to digital signals, then perform digitalprocessing on the resulting digital signals. Data-handling circuitry 508may be configured to provide additional data operations on the digitalresults of ADCs within the array 500. Serial output taps 510 may beemployed to send the processed signals, for example, to an analyzer forcomputation and analysis. The analyzer may take the form of a coremicroprocessor, a microcontroller, or a general purpose or specialfunction computer, for example.

The block diagram of FIG. 6 illustrates an ADC implementation 600 suchas may be employed within each of the cells 502 of an ADC array 500 inaccordance with the principles of the present invention. In thisillustrative embodiment, a photodiode 602 generates a current inresponse to impinging electromagnetic radiation. An amplifier 804, whichmay be, as previously described, a direct injection, buffered directinjection, source-follower, or transimpedance amplifier, amplifies thephotocurrent produced by the photodiode. A voltage to frequencyconverter 808 converts the signal to a digital frequency signal and acounter 808 counts the digital frequency signal. In this illustrativeembodiment, the counter 608 is an up/down counter. The combination oforthogonal transfer capability, up/down counting, and the ability toselect integration times (for example, simply by reading the ADC resultsat intervals of interest) provides the elementary functions required fordigital signal processing. By controlling the accumulation time, thenumber and direction of counts, and the number and direction of shifts,the ADC-array itself may be employed as an array-wide digital signalprocessor.

Using these elementary operations, digital signal processing functionsincluded within the ADC array may include digital filtering, such asspatial or temporal filtering, autonomous digital threshold detection,time-domain filtering, including high-pass or low-pass filtering, anddata compression, using, for example, Decimation. In an illustrativeembodiment, the up/down counter 608 is a linear feedback shift registerthat is configured to perform both counting and data transferoperations. The linear feedback shift register is configured to eitherincrement or decrement the sequence provided by the voltage to frequencyconverter within the same cell, or shifted info the cell from anotherADC cell under control of signal that may foe provided locally(“on-chip” in a single integrated circuit implementation) or remotely(“off-chip,” which could be, for example, on an accompanying controllerin a hybrid implementation, for example).

In an illustrative embodiment, an ADC array in accordance with theprinciples of the present invention may be configured to accept andconvert, analog signals that are spatially mapped to the arrangement ofADCs within the array. The spatial mapping may be, for example, aone-to-one mapping, with signals arriving at the top left ADC within thearray originating at a corresponding location within an array ofsignals, the signal arriving at the bottom right ADC within the arrayoriginating at a corresponding location within an array of signals, andso on. In an integrated circuit embodiment, an entire ADC array may beimplemented using a silicon CMOS process, for example. A digital focalplane array in accordance with the principles of the present invention,one that employs an ADC array in accordance with the principles of thepresent invention, may be a monolithic integrated circuit device, withdetectors and readout integrated circuit formed in a single device, orit may be implemented as hybrid device, with the array of amplifiers,voltage to frequency converters, and counters all implemented in asingle integrated circuit (using Silicon CMOS technology, for example)and mated, with a photodetector array using, for example, bump bonding.In such an illustrative embodiment, one in which an ADC array inaccordance with the principles of the present invention is employed as areadout integrated circuit that operates in conjunction with aphotosensor array, each of the ADCs within the array may occupy no morearea than the area consumed by each of the corresponding photosensors.

In an illustrative embodiment an all-digital readout integrated circuitin accordance with the principles of the present invention may be usedin conjunction with a cryogenically cooled infrared detector array, withconnections between the detector array and the ROIC made via indium bumpbonding. The hybrid device thus formed is referred to herein as adigital focal plane array. In an illustrative embodiment, the detectorarray senses incoming optical radiation in the infrared region of thespectrum (2-20 microns) using photodiodes to create currents that areproportional to the optical radiation impinging on the photodiodes. Thatis, each photodiode (also referred to herein as a pixel) in the detectorarray produces a current that is proportional to the photon fluximpinging upon it. Each photodiode in the array has associated with it aunit ceil in the ROIC. The current in each photodiode is collected inthe photodiode's associated unit cell within the ROIC. The unit cellelectronics integrate the charge and produces, via an analog to digitalconverter (ADC), a digital number (DN) that is proportional to the totalcharge accumulated over the frame period. In this illustrativeembodiment, the DN for each pixel is then shifted to the edge of theROIC and multiplexed with oilier DNs associated with other pixels forserial transfer off the array. By digitizing the signal whilephotoelectrons are being collected, rather than after chargeaccumulation, the need for large charge storage capacitors and highlylinear analog electronics can be eliminated. The power dissipation andnoise problems associated with a conventional, analog readout, approachare also greatly reduced. Additionally, this approach permits operationwith circuitry that operates from a lower level power supply, becausethe dynamic range requirements associated with conventional systemsneedn't be maintained. Permitting operation with lower-level powersupplies permits the use of Integrated Circuit processes that offer muchsmaller feature sizes, thereby further enabling the ADC and readoutcircuitry to be packed within an area less than or equal to the areaconsumed by the associated photodiode. Simplifying the unit cellpreamplifier offers considerable power savings for large arrays.

In this illustrative embodiment, the capacitor is sized to define theleast significant bit of the ADC. In this way, the size of the capacitormay be kept to a minimum, thereby significantly reducing the arearequired for the analog to digital conversion. In this illustrativeembodiment, the analog to digital conversion is achieved via avoltage-to-frequency converter in which a predetermined amount ofphotocurrent charges the capacitor to a level that produces an outputpulse and resets the capacitor. The output pulses are counted and thecount in a given time period corresponds to the amount of photocurrentand, correspondingly, the light flux impinging on the associatedphotodiode. In this way, the illustrative embodiment of a DFPA inaccordance with the principles of the present invention, digitizes thesignal while photoelectrons are being collected, rather than aftercharge accumulation.

A system and method in accordance with the principles of the presentinvention may be employed to form a DFPA that includes a Nyquist-rateADC formed wholly within the area of the ADC's associated detector or,pixel, pitch. In such and embodiment, each of the ADCs may operateindependently of the other ADCs associated with other photodiodes.

The block diagrams of FIGS. 7A and 7B provide a more detailedillustration of data handling structures, such as structures 508 of FIG.5, in accordance with the principles of the present invention. In FIG.7A a fast shift register 700 may be employed to burst data at a highrate as the data is fed into the fast shift register 700 from slower rowshift registers 702. In accordance with the principles of the presentinvention, the data handling structures may also perform various logicoperations may on the data. Logic operations may be used, for example,for thresholding data and match filtering. Power consumption and datatransmission off-array may be greatly reduced by employing a thresholddigital value 704, comparing 706 data from the fast shift register 700to the threshold value, and transmitting only values that meet thecomparison requirements (for example, greater than, less than, or equalto). As illustrated in the block diagram of FIG. 7B, a parallel toserial multiplexor 708 may be employed to serialize the low-rate data toa high-rate output tap.

An ADC array in accordance with the principles of the present inventionmay be particularly well-suited for use in an imaging system. The blockdiagram of FIG. 8 illustrates the components of an imaging system 300 inaccordance with the principles of the present invention. In thisillustrated embodiment, a signal gathering mechanism 802 feeds analogsignals to a detector array 804. In an optical imaging embodiment, thesignal gathering mechanism 802 may include optical elements, such aslenses and the detector array may include an array of photodiodes, forexample, in chemical detection or analysis system, or biological detector analysis systems the signal gathering and detector systems may beoptical filter-, dispersive element-, or interferometer-basedspectrometers. The illustrative imaging system 800 employs an ADC arrayin accordance with the principles of the present invention in thereadout integrated circuit 806 in conjunction with the sensor array 804,to gather and process analog signals. The analog signals subject toprocessing in such an implementation may represent electromagneticradiation characterized by any wavelength, and need not be limited tosignals that might conventionally be associated with “imaging.” That is,in addition to signals that represent phonon flux, the signals may,directly, or indirectly, represent chemical or biological content, forexample. Such an imaging system may employ an ADC array in accordancewith the principles of the present invention to, in addition toconverting analog signals to digital signals, perform digital signalprocessing operations on the converted signals. Further computations andanalysis may be performed by the imaging system controller 808 on theoutput of the ADC array. The controller 808 may also be referred toherein as an analyzer.

The controller 808 may be implemented in a variety of ways and may takethe form of a single-chip microcontroller, a mainframe computer, oranything in between. The controller may be configured to initiate thetransfer of digital signals from the readout integrated circuit 808 andto control the shifts, counts, and accumulation times that permit areadout integrated circuit in accordance with the principles of thepresent invention to perform digital signal processing functionson-chip. Control functions may also be incorporated in a readoutIntegrated circuit in accordance with the principles of the presentinvention, or in a hybrid implementation. Such an integration wouldpermit the ADC array to operate autonomously and to perform on chip-DSPfunctions relatively free of external control. An imaging system 800 inaccordance with the principles of the present invention would beparticularly suited to use in industrial inspection, surveillance,process control, biological research, chemical research,pharmaceuticals, medical imaging, remote sensing, and astronomy, forexample. A compact implementation, using a single-chip readoutintegrated circuit in conjunction with camera controller would beparticularly well-suited to operation within a variety of cameras,including portable consumer still and motion cameras, for example. Aspreviously described, the density of the sensor (e.g., photodiode) arrayis not limited by the ADC and associated circuitry at each pixellocation. Consequently, extremely high-resolution imaging may beachieved by a digital focal plane array in accordance with theprinciples of the present invention.

As previously described, an ADC array in accordance with the principlesof the present invention may be employed by a digital focal plane arrayto incorporate on-chip processing to further enhance the digital focalplane array's performance and, by extension, the performance of animaging system 800. Such processing may include: digitalspatial/temporal filtering that may be implemented by pixel binningwhereby image resolution may be traded for dynamic range and/or framerate; autonomous digital threshold detection that may be used in launchdefection or flash detection, for example; time domain digitalfiltering, such as high-pass or low-pass filtering that may be employedto identify fast- or slow-moving objects; time domain image detectionand jitter compensation that, for example, that shifts digital row datasynchronously the camera line of sight (LOS) and provides imagestabilization for field of view jitter, data compression to reduce rawdata rate using, for example, decimation; and the use of edgeenhancement, such as octonet edge enhancement to aid in featureidentification.

A imaging system 800 that employs a digital focal plane array inaccordance with the principles of the present invention may operate in avariety of modes. In an unmanned aerial vehicle (UAV) application, forexample, such a sensor may operate in: a high-resolution day/nightpanchromatic surveillance mode; a low resolution day/night rapidpanchromatic battlefield monitoring, flash/launch detection mode; a towresolution day/night hyper-spectral effluence classification andatmospheric condition mode; and a time domain imaging (TDI) mode for“push broom” terrain mapping. An apparatus and method in accordance withthe principles of the present invention may be particularly well-suitedfor use in a large format digital focal plane array (DFPA). Such a DFPAcould provide high-resolution wide-area coverage and would thereby findapplication in air and space intelligence, surveillance, andreconnaissance (ISR). In such an application the DFPA could drasticallyreduce the number of assets required for global persistent day/nightsurveillance and launch detection and fully utilize diffraction-limitedresolution. The DFPA could also be employed in atmospheric monitoring,providing the ability to detect the initiation of severe weather eventson a global scale, thereby lengthening warning times that might allowthose in the paths of storms to avoid or ameliorate disaster. A digitalfocal plane array in accordance with the principles of the presentinvention may also be used in the defection and identification ofchemical agents, for example, in the search for weapons, includingweapons of mass destruction. Such detection and identification ofchemical agents may also be employed in chemical, medical, or industrialsystems in large-scale industrial inspection and massive parallelmolecular process monitoring applications. An imaging system 800 thatincorporates an ADC array in accordance with the principles of thepresent invention may be configured to operate, for example, as a camera(in any of a number of electromagnetic bands, including, but not limitedto, the visual and infrared) as a surveillance system, as a roboticcontrol system, or as a chemical or biological detection oridentification system.

The flow chart of FIG. 9 provides an illustration of the process wherebya ADC array in accordance with the principles of the present inventionprovides signal processing functions on-chip. By utilizing thepreviously described orthogonal transfer and bi-directional countingfeatures of the array, many digital signal processing algorithms may beimplemented directly on the array (on an imaging chip in a digital focalplane array embodiment) prior to reading any data out of the array.Conventional focal plane arrays must read ail data out of the array andprovide the data to a processing unit (also referred to herein as acontroller) to perform image processing task. In the example of FIG. 9,an ADC array in accordance with the principles of the present inventionperforms a convolution operation. This can be accomplished bymanipulating the integration time, count shift position on the array,and ADC counting direction. The sign is controlled by the countdirection. The extent of the convolution kernel is defined by the numberand direction of transfers between each integration period. Thefollowing kernel can be implemented by following the steps of the flowchart of FIG. 9.

The Kernel:

-   -   −1 −1 −1    -   −1 8 −1    -   −1 −1 −1

The process begins in step 900 and proceeds from there to step 902 wherethe photocurrent is integrated for eight time units and each counter inthe ADC array is Incremented by the number of counts corresponding tothe photocurrent that is integrated during the eight time units. Fromthere the process proceeds to step 904, where all array count values areshifted to the left one pixel. Circular shifting at the edge of thearray is optional. In step 908 the photocurrent is integrated for onetime unit and the counters are decremented by the number of countscorresponding to the photocurrent that is integrated during the one timeunit, in step 908 all array count values are shitted up one pixel (thatis, a column shift to the nearest neighboring ADC “above”). In step 910the photocurrent is integrated for one time unit and the counters aredecremented by the number of counts corresponding to the photocurrentthat is integrated during the one time unit, in step 912 all array countvalues are shifted to the right one pixel. In step 914 the photocurrentis integrated for one time unit and the counters are decremented by thenumber of counts corresponding to the photocurrent that is integratedduring the one time unit. In step 916 all array count values are shiftedto the right one pixel in step 918 the photocurrent is integrated forone time unit and the counters are decremented by the number of countscorresponding to the photocurrent that is integrated during the one timeunit, in step 920 all array count values are shifted down one pixel. Instep 922 the photocurrent is integrated for one time unit and thecounters are decremented by the number of counts corresponding to thephotocurrent that is integrated during the one time unit. In step 924all array count values are shifted down one pixel. In step 926 thephotocurrent is integrated for one time unit and the counters aredecremented by the number of counts corresponding to the photocurrentthat is integrated during the one time unit. In step 928 all array countvalues are shifted to the left one pixel. In step 930 the photocurrentis integrated for one time unit and the counters are decremented by thenumber of counts corresponding to the photocurrent that is integratedduring the one time unit. In step 932 all array count values are shiftedto the left one pixel. In step 933 the photocurrent is integrated forone time unit and the counters are decremented by the number of countscorresponding to the photocurrent that is integrated during the one timeunit. In step 934 circular shift is disabled (if previously enabled) anddata may be read out. The process proceeds to end in step 938. Theseoperations effectively convolve the kernel with the entire image priorto readout.

Employing the integrated digital signal processing of an array inaccordance with the principles of the present invention, a variety offunctions may be performed within the array itself. For example, in aspectral correlation application, grating-based instruments typicallyuse an imaging array to detect the dispersed spectrum of an image scene.There is a spatial end a spectral dimension to the data gathered by anarray in such an application. In accordance with the principles of thepresent invention, cross correlations to known spectra can be calculatedby manipulating, shifts, count direction, and integration times, asillustrated In the example of FIG. 9.

In image processing applications, filters for edge detection, smoothing,differentiation, etc., can be performed by an array In accordance withthe principles of the present invention using the shifts, adds, andaccumulations in manner similar to the process of FIG. 9. In such anapplication, the scene must remain stationary for the entire timerequired to implement the filter. Similarly, match filtering may beimplemented by developing a filter kernel for cross correlation ofobjects In the scene (e.g., a person's iris in a personal identitysystem). The correlated image can then be thresholded by compare logicfor detections. The array could readout a raw cross-correlated imago, ordetections atone, in another aspect of the array, a steering mechanism,which may be external, may be employed to shift accumulated counts in anarray in accordance with the principles of the present inventionsynchronously with the overall motion of an imager and, thereby, permittime domain integration. Similar control of shifting by the output of asensor inertial measurement unit (IMU) allows an array in accordancewith the principles of the present invention to perform imagestabilization. Non-uniformity correction may be performed directly on anarray in accordance with the principles of the present invention bycounting down for a frame period while viewing a flat, extended source.Once the background scene is acquired, the counters are configured tocount up, and the scene data is acquired. After a frame period, onlysignal and noise data will be present; non-uniformity correction willhave been achieved. This technique employs a 50% duty cycle for sceneand background looks.

An array in accordance with the principles of the present invention mayalso be employed to form an optical central processing unit. In such anapplication, an array of light sources, such as LEDs can be imaged ontoa digital focal plane array in accordance with the principles of thepresent invention for computations in the optical domain. The LEDs maybe driven, for example, to form the appropriate signals, and the focalplane array can be controlled to add, subtract, multiply and divide, asdesired. These operations may be accomplished by controlling theintegration time, the count shift position on the array and the ADCcounting direction, as described in the discussion related to FIG. 9. Ina large array implementation, such a CPU could perform massivelyparallel operations.

An imaging system in accordance with the principles of the presentinvention may be configured for operation as a camera, with manyapplications in various light-sensing ranges. In the visible range, sucha camera may fake the form of a low-cost still or video camera, ahigh-end professional-grade video camera, or anything in between.

In the infrared range, an imaging system in accordance with theprinciples of the present invention may be configured for operation asan infrared camera in industrial sensing, security, or firefighting, forexample. Such a camera may also be employed to provide an infrared imageof the mad ahead of a vehicle. The infrared image may be displayed, forexample in a “heads up” display for viewing by the vehicle operator. Ifthe infrared image extends beyond the vehicle's headlights, the vehicleoperator may be alerted, for example, to the presence of deer or otherhazards on the highway ahead beyond the range of the headlights.

Additionally, an infrared camera in accordance with the principles ofthe present invention may be configured as a chemical-imaging camera.Such a camera may be employed to capture the chemical composition anddistribution of a sample, for example. Such a camera may be employed inmilitary, research, industrial, security, and fire-fightingapplications. Such a camera may be employed to enable a biohazard teamto quickly determine the composition of spilled material from a safedistance. By coupling a Fourier Transform Infrared spectrometer to adigital focal plane array in accordance with the principles of thepresent invention, a camera may be employed to produces images thatinclude both spectral and spatial information. Such a combination ofFTIR spectrometer with focal plane array in accordance with theprinciples of the present invention enables the collection spatial,spectral, and intensity Information simultaneously, yielding a pictureof the chemicals that make up a sample and their distribution within thesample.

The foregoing description of specific embodiments of the invention hasbeen presented for the purposes of illustration and description, it isnot intended to be exhaustive or to limit the invention to the preciseforms disclosed, and many modifications and variations are possible inlight of the above teachings. The embodiments were chosen and describedto best explain the principles of the invention and its practicalapplication, and to thereby enable others skilled in the art to bestutilize the invention. It is intended that the scope of the invention belimited only by the claims appended hereto.

What is claimed is:
 1. An apparatus comprising: a sensor arraycomprising a sensor to generate an analog signal; an analog-to-digitalconverter (ADC) array comprising an ADC operably coupled to the sensor,the ADC comprising a capacitance to be charged by the signal, thecapacitance defining a least significant bit (LSB) of the ADC andconfigured to convert the analog signal into digital data; and transfercircuitry, operably coupled to the ADC array, to transfer the digitaldata among ADCs in the ADC array synchronously with motion of an imageacross the sensor array, wherein the ADC comprises a counter to store atleast a portion of the digital data as a digital number and to incrementand/or decrement the digital number in response to digital datatransferred from another ADC in the ADC array.